Image‐level dataset synthesis with an end‐to‐end trainable framework

Dataset synthesis via virtual engines like Unity is attracting much more attention in recent years due to its low cost at obtaining ground-truth labels. For this kind of work, environments are constructed within the engine mimic real-world, either with great manual efforts or learning-based methods. The latter shows superiority over former when target real-world scenes changeable, from which attributes can be automatically adjusted based on distribution difference between synthetic and datasets. However, non-differentiability whole pipeline hinders efficiency attribute optimization. To end, paper proposes simulate datasets a fine-grained perspective, such that system trained an end-to-end manner. Specifically, it converted into image-level data problem, designs constraint using content loss two images. As rendering process mathematically unknown, blocks back propagation gradients, generative model approximate engine. result, framework becomes fully differentiable optimized efficiently by gradient descent. Experimental result our method useful training Besides, found enables learn potential data, hard achieved existing far as we know, first attempt finish task process.